Dual-mode eyeglasses

ABSTRACT

Methods, systems, and techniques for dual-mode communication are provided. Example embodiments provide a an enhanced mobile device that includes a CPU, a low latency transceiver, and a high latency transceiver. The enhanced mobile device is in communication with a console via both the low latency transceiver and the high latency transceiver. The console also includes a CPU, a low latency transceiver, and a high latency transceiver. In some embodiments, the low latency communication channel is used to transmit control messages to an entertainment console whereas the high latency communication channel is used to communicate data between the mobile device and the entertainment console. The dual-mode communication may be used in different scenarios, including entertainment applications, such as with dual-mode eyeglasses.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/934,041 filed Jul. 2, 2013; which claims priority from U.S. PatentApplication Nos. 61/667,261 filed Jul. 2, 2012, 61/675,211 filed Jul.24, 2012, and 61/682,668 filed Aug. 13, 2012, the contents of whichapplications are incorporated herein by reference in their entireties.

TECHNICAL FIELD

The present disclosure relates to methods, techniques, devices, andsystems for interacting with an entertainment system and, moreparticularly, to methods, techniques, devices, and systems forinteracting with a gaming system using dual-mode communicationeyeglasses.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example block diagram illustrating dual-mode communicationaccording to an example embodiment.

FIG. 2A compares latency of various communication mechanisms.

FIG. 2B illustrates a combination of low and high latency communicationaccording to an example embodiment.

FIG. 2C illustrates advanced gaming scenarios facilitated via dual-modecommunication according to an example embodiment.

FIGS. 2D and 2E illustrate an example arena gaming environment accordingto an example embodiment.

FIG. 2F illustrates example dual-mode eyeglasses according to an exampleembodiment.

FIGS. 3.1-3.8 are example flow diagrams of processes performed byexample embodiments.

FIG. 4 is an example block diagram of an example computing system forimplementing example embodiments.

DETAILED DESCRIPTION

The current mobile device environment lacks any kind of fast/low latencywireless connection to entertainment systems, such as gaming consoles,set-top boxes, or the like. This deficiency stands in the way of thegoal of providing a seamless wireless connected experience between smartphones (and other mobile devices) and home entertainment systems.Latency is a measure of time delay experienced in a communicationsystem. Latency may be measured as the time from the beginning of atransmission (e.g., a signal, packet, etc.) to the beginning of thereception of the transmission. Latency is related to response time, as alow latency connection will typically result in a lower response timeand thus more interactivity. Accordingly, low latency connections aretypically used or preferred where fast, highly interactive control iscritical, like in a game controller. But low latency connections mayhave lower range (e.g., 2.4 GHz and/or Bluetooth) and/or provide lessbandwidth to transmit or receive data. High bandwidth data connections,like Wi-Fi, have greater latency and do not work effectively whenemployed in scenarios where a game controller would be used because ofthe much greater latency. Higher bandwidth connections may be utilizedwhere the amount or volume of data being sent is larger, as inmegabytes, and gigabytes, but not needing low latency/faster responsetimes.

The techniques described herein are based on dual-mode console/devicecommunication. In some embodiments, a mobile device (e.g., a smartphone, tablet computer, smart glasses) is enhanced and/or configured toinclude a low latency transmitter and a high latency transceiver. Theenhanced mobile device can then be used to transmit via the low latencytransmitter control signals (e.g., gaming commands such as up/down,left/right) to an entertainment console (e.g., a game system, set-topbox). Typically, the low latency transceiver is short range and lowbandwidth, such as 2.4 Ghz game controller communication protocols,Bluetooth, infrared, or the like. Note that in some embodiments, the lowlatency transmitter may be, or be part of, a low latency transceiver,such that bidirectional low-latency communication with the console ispossible. However, it is not necessary for the enhanced mobile device tohave the capability to receive data over a low latency connection.

In general, a low latency communication includes any connection having alatency that is low enough to support a particular interactive gaming orentertainment experience. Thus, “low latency” may depend in part on theapplication or setting. For an interactive, real-time video game, alatency of up to 40 ms may be acceptable (while some games and users mayrequire even lower latencies, such as 10 ms, 15 ms, 20 ms, or the like).For channel surfing or menu navigation, a latency of between 50 and 100ms may be acceptable to provide an interactive experience. In addition,latency may be measured with respect to the communication link (e.g.,the time taken for the first bit of data to arrive at the receiver),while excluding processing time by the application. If such ameasurement is used, then a lower latency connection may be required, toaccount for application processing time. Thus, if the applicationprocessing time is 20 ms, then a maximum acceptable latency of thecommunication link may be lower (e.g., 20 ms to yield a total latency of40 ms that includes application processing time of 20 ms).

Concurrently, the mobile device can communicate data (e.g., game data,image data, video data, audio data) with the entertainment console viathe high latency transceiver. Typically, the high latency transceiverprovides higher bandwidth than the low latency transmitter. For example,the high latency transceiver may communicate via IP over a Wi-Fi, 3G, or4G (e.g., LTE) connection.

Using the described techniques, seamless entertainment becomes possiblebetween a mobile device and a console. The techniques can provideexperiences such as playing the same game on both a console and smartphone, controlling streaming movies on a TV from a smart phone, andtransferring movies from a TV to a smart phone (and vice versa)midstream, and the like.

Note that many of the examples herein are based on a mobile device thatis a smart phone enhanced to perform dual-mode communication. However,in other embodiments, other mobile devices may be similarly enhanced.Other example mobile devices include, but are not limited to, tabletcomputers, eBook readers, smart glasses (e.g., eyeglasses or gogglesthat include a display mechanism), and display-enabled helmets.

Introduction & Overview

FIG. 1 is an example block diagram illustrating dual-mode communicationaccording to an example embodiment. FIG. 1 illustrates an enhancedmobile device that includes a CPU, a low latency transceiver, and a highlatency transceiver. The enhanced mobile device is in communication witha console via both the low latency transceiver and the high latencytransceiver. The console also includes a CPU, a low latency transceiver,and a high latency transceiver. Note that in some embodiments, the lowlatency communication between the enhanced mobile device and the consoleis one-way or unidirectional. That is, the enhanced mobile device mayonly be able to transmit to (and not receive from) the console via thelow latency communication connection/link. Furthermore, the low latencyconnection may not rely on handshaking or similar operations, such asmay be required to set up or tear down a TCP/IP connection.

Dual-mode wireless communication according to some embodiments includesat least some of the following features:

-   -   Utilize existing console 2.4 GHz wireless connection or        Bluetooth schema for control commands    -   Typical control commands may include existing console controller        commands such as: dual analog stick movement (up/down, left        right, push), d-pad (8 way digital control pad), action input        buttons, adjustable analog triggers, start and back buttons,        power or command button.    -   Device control commands, sensors, or input devices become        available to the console. These console control commands may        include: smart phone touch screen and (soft) keyboard inputs,        tilt, bank, motion (e.g., accelerometer, magnetometer,        gyroscope), compass/direction, GPS/AGPS, camera aiming. Also,        future device control commands similar to Microsoft's Kinect        sensor could be built into a device for 3D gaming.    -   Feedback from the console sent back to the enhanced mobile        device over the low latency connection may include low response        time feedback such as a vibration signal sent to the device        (e.g., to signal when the user collides with an object in a        driving game).    -   Utilize Wi-Fi, 3G, 4G (e.g., LTE), or future data connections        for high bandwidth data transfer.    -   Example uses for data transfer include: 2nd (or 3rd, 4th, etc.)        screen display for a game being played where the 1st screen may        be a TV connected to a console (or vice versa), metadata        relevant to a game being played on the main console connected        TV, soft controls (graphically display on the device screen)        that change relevant to a level of a game, options available at        certain points in a game, a game map that updates as the player        moves, metadata relevant to a movie or other video program being        watched on the main screen, mobile user profile storage, and        also used to map an area around the user by utilizing the device        camera to input the visual setting around the user. Voice        commands or other input can also be transferred via data to the        console.

In some embodiments, low latency communication is provided via a 2.4 GHzand/or Bluetooth connection schema or protocol. For example, in the caseof Microsoft Xbox 360, a proprietary 2.4 GHz game controller protocol isused. For Nintendo Wii and Sony PS3 consoles, Bluetooth is utilized. Theterm “Bluetooth” may include current and future versions of theproprietary open protocol for creating short range personal areanetworks, communicating in the 2.4-2.48 GHz range, using frequencyhopping spread spectrum techniques utilizing 79 channels of 1 MHz each.In some cases error checking (e.g., Extended Synchronous Connections)may be utilized, although in some embodiments some error checkingprotocols may not be used to reduce latency.

Some embodiments include multiple low latency transmitters (ortransceivers). For example, the enhanced mobile device may include botha Bluetooth transceiver and an infrared transmitter. In such cases, theenhanced device may include logic to determine which of the multipletransmitters to use. For example, the enhanced device may preferentiallyutilize one transmitter until it has reached or is near its capacity,and then transmit overflow data via the other transmitter. As anotherexample, the enhanced device may seek to divide or balance thetransmission load between the two (or more) transmitters. In someembodiments, the choice of transmitter to use may be exposed to the gameapplication or other software, so that an application developer maycontrol which transmitter to use.

In some embodiments, control commands (and other signals, messages, ordata needing a low latency connection) are separated from more dataintensive examples (mentioned above) by logic executing on the CPU.Control commands are converted to protocol format for transfer acrossthe 2.4 GHz game controller or Bluetooth band. The enhanced mobiledevice then employs its 2.4 GHz game controller or Bluetooth transmitterto send a signal in the protocol format specified. On the other end, theconsole receiver receives radio waves emitted by device receiver and thedata in protocol is converted by CPU to perform commands with respect toan application or other function/code performed by the console. Inreverse, if the console needs to send fast, low latency data to theenhanced mobile device, it can do so over the same connection as boththe device and console may have transmit and receive functionality.

In some embodiments, when the enhanced mobile device or console CPUdetermines or identifies data that is of a non-control type suitable fortransmission by a higher latency, higher bandwidth connection likeWi-Fi, LTE, or 3G, (or a future high bandwidth data connections), itconverts the data to the specified protocol format required for transferacross the higher bandwidth connection. This higher bandwidth data isreceived by the corresponding high latency receiver and employed toperform higher bandwidth, higher latency activities of the typedescribed above.

FIG. 2A compares latency of various communication mechanisms. Inparticular, FIG. 2A compares the latency of example wire-basedcommunication between a game controller and a game console (0.3 ms) andexample direct 2.4 GHz wireless communication between a game controllerand a game console (4-8 ms). While the latency of direct wirelesscommunication is greater than that of wire-based communication, it isstill low enough to play highly interactive video games.

FIG. 2A also compares the latency of various indirect, high latencycommunication approaches between a mobile device (e.g., smart phone) anda game console. The compared approaches all utilize a Wi-Fi or cellularnetwork in addition to various intermediary devices (e.g., a wirelessrouter) or systems (e.g., cellular network). In particular, FIG. 2Acompares the latency of communication between a mobile device and a gameconsole via a Wi-Fi network, an LTE network, and a 3G network. In thisexample, Wi-Fi provides the lowest latency of the three approaches—about53 ms in the best case (the sum of 3 ms from smart phone to wirelessrouter, 25 ms from router to intermediary system, 25 ms fromintermediary system to game console). However, this best-case latencyfar exceeds that required to provide an enjoyable interactive gamingexperience.

FIG. 2B illustrates a combination of low and high latency communicationaccording to an example embodiment. FIG. 2B shows how low and highlatency communication may be combined in example embodiments. Inparticular, low latency control-oriented communication is performed viaa 2.4 GHz mechanism, and high latency data-oriented communication isperformed via a Wi-Fi or LTE network. By incorporating bothcommunication mechanisms in a single enhanced mobile device, theenhanced mobile device may be employed to provide an enjoyable,interactive gaming or viewing experience.

FIG. 2C illustrates advanced gaming scenarios facilitated via dual-modecommunication according to an example embodiment. In particular, FIG. 2Cillustrates game or program portability, new controller paradigms (e.g.,using advanced sensors of mobile devices, such as accelerometers andgyroscopes), and remote touch screens (e.g., touching a screen of amobile device is translated to a “touch” or input to a remote televisionor other display).

The described techniques may bring a new level of precision and accuracyto gaming applications. Current tilt-based gaming uses only theaccelerometer, resulting in an imprecise record of motion, which whilestill fun, is noticeably coarse in the level of control afforded.However, with the addition of a gyroscope together with high-precisionsensor fusion and calibration, games can be controlled in a more preciseand granular fashion, enabling more advanced tilt- and twist-basedgames. This has proved popular with both advanced and casual golfers, asadvanced motion can track the speed and arc of your swing, as well asthe angle of the club face, and is therefore able to provide a veryaccurate replication of your golf shot, with distance, elevation hookand slice all accounted for accurately. In addition, advanced motiontechnology can enable your phone to be used to control game consoles andother gaming devices.

Arena Gaming

FIGS. 2D and 2E illustrate an example arena gaming environment accordingto an example embodiment. In some embodiments, enhanced mobile devicesas discussed herein may be used by multiple users to engage ininteractive, immersive, real time, multi-player video game experiences.

The advent of motion controllers and motion sensors in console, PC andmobile gaming has transformed entertainment from a static, “sit on thecouch” environment to an interactive movement-based paradigm. The user'smovements are sensed by the platform (e.g., console, PC, mobile) andtranslated into movements displayed by entertainment software orapplication on a screen. Jumping, dancing, lunging, waving arms, and thelike, are all sensed and interpreted into movement in a game orapplication.

The existing gaming paradigm typically includes one of more usersinteracting directly with the TV or device via the methods above. Whatis absent is the ability for users who are proximately situated (e.g.,present in the same room or other enclosed space) to interact with eachother. For example, given a first and second player who are in the sameroom, there is no way for the player to shoot at, throw a ball to, orotherwise direct game play towards, the second player by aiming acontroller at the second player.

By combining motion sensing and/or 3D sensing functionality withenhanced mobile devices having dual-mode communication capabilities aswell as enriched sensors, such as cameras, gyroscopes, compasses, andthe like, the concept of multi-user arena gaming becomes possible. Thetechniques described below combine motion/3D sensing with enhanceddual-mode mobile devices to provide an arena gaming environment wheremany users can engage in game play by interacting with the environmentaround them as well as each other.

Enhanced dual-mode mobile devices facilitate an interactive, real-timearena gaming experience for multiple players due to their advantageouscommunication properties. In particular, a dual-mode mobile device canuse its low latency connection to communicate with a gaming system(e.g., console or PC) and/or with other proximately located mobiledevices. The low latency connection may be used to communicate sensordata (e.g., from an accelerometer) to the gaming system and/or anothermobile device.

In some embodiments, adding 3D/motion sensing to the enhanced mobiledevice allows a user to “see” the other user's motions. For example, anenhanced mobile device that includes motion sensing may be operated byuser A to target user B, even when user B does not operate an enhancedmobile device, because user A's mobile device alone is capable ofsensing user B's position and movements. In other scenarios, of course,both users A and B may operate similar enhanced mobile devices. Inaddition, by situating motion sensing on a mobile device, players neednot be in proximity to a motion sensor associated with a game console orother gaming system. For example, they may be in a different room.Furthermore, situating motion sensing on a mobile device may reducelatency, because the mobile device need not wait to receiveposition/motion information from a remote system.

Note that while several of the examples described herein are based ontwo players or gamers, typical embodiments are capable of facilitatingarena gaming amongst more than two players. Also, some embodiments mayinclude multiple motion sensing and/or 3D sensing devices to create alarger arena area than would be covered by a single sensor. In general,arena area may be expanded or enlarged by adding additional sensingdevices. Each sensing device may communicate (e.g., via wirelessconnection) information about its local area back to the console or PCthat is managing the game. Furthermore, some embodiments may be able tolink multiple distinct arenas to provide a distributed gamingexperience. For example, a first arena (e.g., a living room in a firsthouse) hosting a first group of three players may be linked with asecond arena (e.g., a gymnasium) hosting a second group of six players,such that players of the first group can interact with each other aswell as players of the second group, and vice versa.

FIG. 2D depicts one arena gaming example. In the illustrated example, auser operating Device A is interacting with another user operatingDevice B. Various actions, such as a dive (to avoid a shot fired atthem), a sideways movement (to avoid a punch), a throw (say of a handgrenade), a catch, a throw, or the like, are all sensed by Device A's3D/motion sensing features. Combined with Device A's onboard mobiledevice capabilities; touch screen and (soft) keyboard, tilt, bank,motion (e.g., accelerometer, magnetometer, gyroscope),compass/direction, GPS/AGPS, camera aiming, it is possible to haveDevice A “lock on” to User B with a scope and fire at them veryaccurately.

Mobile devices in typical form factors (e.g., smart phone, pad or smartcontroller) have immediate immersive arena entertainment possibilities.New mobile form factors, such as goggles, provide an even more immersivearena experience where the user is looking through a screen to sight atarget and head or eye movements are translated via dual-modecommunication functionality to a PC, console, or other mobile device orgaming system.

In one embodiment, arena gaming utilizes one or more enhanced mobiledevices communicating sensing and translating movement of both themobile device, other potential users (targets) or the environment toform an immersive arena experience. A CPU on one or more of the device,the console/PC, or cloud software (or combination of) then interpretsthe controls below and provides feedback to the user via graphics,sound, vibration in a software program running on the PC/Console, mobiledevice or cloud. Some embodiments of arena gaming utilize the hereindescribed dual-mode communication functionality to immerse the user.

Typical control commands include existing console controller commandssuch as: dual analog stick movement (e.g., up/down, left right, push),d-pad (e.g., 8-way digital control pad), action input buttons,adjustable analog triggers, start and back buttons, power or commandbutton. These device control commands/inputs may also become availableto the console, the cloud, and/or other gaming systems. These consolecontrol commands may include one or more of: smart phone touch screenand (soft) keyboard, tilt, bank, position/orientation/motion(accelerometer, magnetometer, gyroscope), compass/direction, GPS/AGPS,camera aiming, and the like.

As noted the low-latency connection may be used between a console and amobile device and/or between multiple mobile devices themselves.Feedback from the console sent back to the device over the low latencyconnection typically includes low latency feedback such as a vibrationsignal sent to the device when the user might hit something when drivinga car in a game. Control commands or sensor inputs from one device mayalso be directly shared with other mobile devices, so that other mobiledevices can accurately represent or render the position, orientation, orother properties of other users/devices.

In addition, arena gaming may utilize the dual-mode capability ofenhanced mobile devices for high bandwidth data transfer. Typical usesfor high bandwidth data transfer include: 2^(nd) (or 3^(rd), 4^(th)etc.) screen display for a game being played where the 1^(st) screencould be a TV connected to a console (or vice versa), metadata relevantto a game being played on the main console connected TV, soft controls(graphically display on the device screen) that change relevant to alevel of a game, options available at certain points in a game, a gamemap that updates as the player moves, metadata relevant to a movie beingwatched on the main screen, mobile user profile storage, and also usedto map an area around the user by utilizing the device camera to inputthe visual setting around the user. Voice can also be transferred viadata to the console.

As mentioned above, an arena or other gaming environment may be mappedby a user's camera on a mobile device, where the data would be sent byhigh bandwidth connection to device/console and/or cloud. Also objectsand other users could be mapped by the user's camera in high detail andsent via high bandwidth connection. Once mapped, software present on themobile device can sense changes in the arena environment when input frommotion and depth sensing sensors detects changes.

Embodiments of arena gaming utilize one or more mobile devicescommunicating, sensing, and translating movement of both the mobiledevice, other potential users (targets), and/or objects in theenvironment (e.g., chairs, walls, obstacles, tables, pets) to form animmersive arena experience. A CPU on the device, the console/PC, and/orcloud software (or combination thereof) then interprets the controls andprovides feedback to the user via graphics, sound, vibration in asoftware program running on the PC/Console, mobile device or cloud.

Referring again to FIG. 2D, a surface (x/y) area map is depicted andshows user A depicted by mobile device A. The previously described dualmode high and low latency connections are depicted as streaming to andfrom the device and labeled with “Dual Mode wireless connection.”Streaming of (low latency) control and high bandwidth data can beto/from the cloud, console/PC, or another device via high bandwidthwireless connectivity and/or internet. Note that the presence of thelabel “Dual Mode wireless connection” should not be interpreted to meanthat the use of both low and high latency connections are required asbetween two endpoints in all embodiments. For example, in someembodiments, two mobile devices may only communicate with one anotherusing the low latency connection, may communicate with the cloud usingonly the high latency (high bandwidth) connection, and communicate withthe console using both the low and high latency connection.

In FIG. 2D, a console or PC is associated with a 3D and motion sensor.The motion sensor may in other embodiments be incorporated as part ofthe console or PC. FIG. 2D also depicts mobile devices A and B as inrange of each other and the console. When used in conjunction with aconsole or PC sensor, the movement and motion of device A can be trackedin 3D a given distance from the sensor providing the sensation of theuser moving around a set arena. In some embodiments, multiple consoles,PCs, and/or 3D/motions sensors may be deployed. For example, a house orother residence may be configured to include a 3D motion sensor in eachof the living room, recreation room, and basement, where all of themotion sensors are linked to a common console or PC.

When used without a console or PC sensor (or when players are out ofrange of the sensor), the mobile device(s) may be employed to track eachother's movement and motion relative to each other. As one or more ofthe mobile devices may be in motion, the advantage of the console/PCsensor providing a fixed arena may be lost, and the devices may thenmove relative to each other in a virtual arena. Arena boundaries andobjects that may be mapped in a static arena may become more difficultto track relative to the other device, thus interaction with aconsole/PCs sensor is preferred for some applications.

Mapping and aiming are facilitated by some embodiments. For example, asdevice A is moved around a fixed space on an X/Y/Z axis, this movementis tracked and mapped by either a console/computer 3D/motion sensorand/or the device. As the mobile device is pointed to a direction inspace defined by the X/Y/Z axis, the device's own onboard 3Ddepth/motion sensor is able to map objects in space such as ceilings,wall, furniture, tables, etc. The console/PC tracks the location of thedevice based on both its own 3D/motion sensor and the device's onboardaccelerometer and GPS. The console/PC further tracks the orientation(e.g., direction pointed) of the mobile device, 3D mapping informationsent from the mobile device to the console/PC may be used to accuratelydetermine a model of the arena boundaries, obstructions, and the like.

When additional mobile devices enter the mapped arena, their positionsrelative to objects, boundaries and other mobile devices are accuratelysensed by the console/PC 3D/motion sensor and/or their own onboard3D/motion sensors. The mobile devices can sense changes in the arenaenvironment, based on inputs from motion and depth sensing sensors thatdetect changes. Also, mobile devices can be aimed and/or zoomed at othermobile devices, with information about such actions being shared withother devices or the gaming system. In response, the gaming system canupdate a game model or other representation of player locations andother information about the arena gaming environment.

In addition, some embodiments perform auto-detection of new players whoenter the gaming arena. For example, when a third player enters thearena depicted with respect to FIG. 2D, his mobile device may bedetected by the console. Detection may be based upon the presence ofsignals transmitted by the mobile device of the third player, such as aWi-Fi, Bluetooth, infrared, or other signal. In some embodiments, theconsole may periodically poll for new devices within the arena. When theconsole detects the newly arrived mobile device, it causes the device todisplay a prompt and/or other user interface elements that may be usedby the player to enter or otherwise join the current arena game.

Some embodiments support “smart glasses.” In particular, with the adventof glasses/goggles technology with built in screens, a highly immersivearena experience is provided when leveraging the above techniques in themanner specified. With glasses functioning as the mobile devicesdepicted in the Figures, an arena gaming scenario is readily presented.Additionally adding gun type handheld mobile devices, control inputsfrom the user for aiming and triggering shots, gaming arena scenariosare obvious.

Using the described techniques, some embodiments provide augmentedreality or virtual reality experiences for users. With respect tovirtual reality, an embodiment that uses smart glasses may immerse aplayer in a substitute (virtual) reality, in which the informationpresented via the glasses operates as a replacement for visual signalsfrom the physical environment of the player. With respect to augmentedreality, some embodiments may modify, add, incorporate, or otherwiseaugment views of the physical environment with game-related information,such as indications of friend or foe, weapons or other game objectspossessed by other users, indications of game actions, and the like.

FIG. 2E illustrates additional aspects of an arena gaming embodiment.FIG. 2E illustrates the use of a mobile device to aim or otherwise sightanother user in an arena gaming context. Note that a game user interfaceelement (e.g., cross hairs) is superimposed upon an image/video obtainedfrom the device camera in order to merge the gaming experience withreal-world image data obtained from the camera sensor.

Note that arena gaming embodiments may be deployed in various types ofpublic and/or private establishments or locations. Some embodiments mayoperate in a residential setting, such as within a living room. Otherembodiments may operate in a public commercial setting, such as in alocation (e.g., a warehouse) that has been configured to host arenagaming tournaments. Other commercial settings may include restaurants orbars. For example, a bar may host an arena game for players wishing toengage in a trivia game, games of chance, sporting simulations, or thelike.

In addition, the described techniques need not necessarily be employedin the “gaming” context. For example, at least some of the describedtechniques can be used in a retail sales application. In oneapplication, an “arena” ordering system may be installed in a coffeeshop, restaurant, or other retail establishment. The ordering system maybe to automatically detect the presence of a customer's mobile device(e.g., when the customer enters the establishment) and to transmitordering information (e.g., menu information) and/or controls to themobile device. Then, the customer may use his mobile device to reviewand submit an order from a menu of items available for purchase at theestablishment. The ordering system may include a motion/location sensorthat can detect the location of the customer, so that the customer maysimply seat himself and place his order, without needing to submit theorder at a counter. Then, the ordering system informs a waiter or otheremployee of the customer's location (e.g., table), so that the employeecan make delivery of the requested items.

Dual-Mode Eyeglasses

This section describes wearable computing eyeglasses and accessories,and how those accessories sense and communicate 3D depth, motion, camerasighting and various other control movements typically requiring a lowlatency connection, as well as higher latency/higher bandwidth datatransfer and how those work together to provide a compellingentertainment experience. Wearable computing eyeglasses having dual-modecommunication capability are herein sometimes also referred to as“dual-mode eyeglasses,” “dual-mode glasses,” “enhanced eyeglasses,”“smart glasses,” or similar.

Dual-mode glasses according to some embodiments can be considered anexample of an enhanced mobile device (as described herein), where theuser is able to view through a translucent screen, in the same way aheads up display (“HUD”) typically operates. Viewing data projected on atranslucent or transparent screen is commonly referred to as “augmentedreality.” There are existing augmented reality glasses but theytypically do not contain full mobile device functionality, much less thedescribed dual-mode communication capability.

As with other types of mobile devices, in order for wearable computingglasses to change the face of gaming, arena gaming, or entertainment ingeneral, certain control commands are preferably transferred via lowlatency connections. Other data can be transmitted and received viahigher latency, higher bandwidth wireless connection (e.g., Wi-Fi, 3G,4G, LTE, Wi-Di).

By enhancing eyeglass-based displays/devices with a dual-mode wirelesscommunication solution, seamless entertainment becomes possible betweenthe user and the console, television, PC, or cloud computing services.Dual-mode glasses facilitate experiences such as playing the same gameon both a console (with display) and glasses, controlling streamingmovies on the user's television from his glasses, and porting ortransferring an experience (e.g., a movie or game) from a display (e.g.,television, laptop, personal computer monitor) to the eyeglasses (orvice versa), and the like.

Because the described wireless solution is dual mode—meaning it uses alow latency band for control (e.g., up/down, left/right, forward/back,etc.) and a higher bandwidth (but higher latency) connection for datatransfer—an entirely new connected experience paradigm becomes possiblewhen using dual-mode glasses. For example, when using dual-mode glasses,the user can look through a translucent HUD screen to sight a target, orselect a menu item. In addition, head and/or eye movements may betranslated and then transmitted via dual-mode connections to a PC,console, cloud or other mobile device.

Example dual-mode glasses may utilize existing console, PC or set topbox 2.4 GHz wireless connection or Bluetooth schema for controlcommands, which need to be sent via the low latency connection. Typicalcontrol commands are existing console controller commands such as: dualanalog stick movement (e.g., up/down, left right, push), d-pad (e.g.,8-way digital control pad), action input buttons, adjustable analogtriggers, start and back buttons, power or command button, or the like.Since the glasses themselves lack a console controller stick, d-pad,triggers, and the like, generation of these low-latency consolecontroller commands for transmission can be via various mechanisms,described below.

-   -   Tracking eye movements—by placing a motion sensor facing the        eyeball, the eye movements up/down, left/right may be sensed and        transmitted. Eye blinks and eye movement with eyelid closed        could also be interpreted as above control commands.    -   A soft pad may be attached to the glasses where the user could        utilize up/down, left/right, tap and buttons (similar to a        laptop track pad, or smart phone screen) to generate the above        control commands. Other embodiments may include a trackball or        other type of input device.    -   An accessory device may be used in conjunction with the glasses        to generate the control commands. This device may take the form        of a typical console controller, or even a gun shaped device        (e.g., for action shooter type games).

Since the dual-mode glasses are essentially a mobile computing device,the device control commands become available to the console/PC, cloud orother device. These console control commands include: smart phone touchscreen and (soft) keyboard, tilt, bank, motion (accelerometer,magnetometer, gyroscope), compass/direction, GPS/AGPS, and cameraaiming. The user's head movements while wearing the glasses may besensed via the gyroscopes, accelerometers, magnetometer, and the like.Such movements may also be translated into, and transmitted as, typicalcontrol commands mentioned above.

Feedback from the console/PC or cloud sent back to the glasses over thelow latency connection may include necessary low latent feedback like avibration signal sent to the device when the user might hit somethingwhen driving a car in a game.

Example dual-mode glasses utilize Wi-Fi, LTE, 3G, or future dataconnections for high bandwidth data transfer. Typical uses for datatransfer include: 2^(nd) (or 3^(rd), 4^(th), etc.) HUD type screendisplay on the glasses for a game being played where the 1^(st) screenmay be a TV or display connected to a console (or vice versa); metadatarelevant to a game being played on the main console connected TV; softcontrols (e.g., graphically display on the device screen) that changerelevant to a level of a game; options available at certain points in agame; a game map that updates as the player moves, metadata relevant toa movie being watched on the main screen; mobile user profile storage;camera data obtained from the dual-mode glasses and used to map an areaaround the user based on his visual setting; sighting or aiming a realor virtual device (e.g., a weapon in a game); and 3D and/or motionsensing data obtained by or provided to the dual-mode glasses.

A microphone mounted in the glasses may be used to transfer voice to theconsole. Independent stereo sounds may be transferred to earbuds/speakers on the glasses. In some embodiments, dual-mode glasses mayprovide position dependent surround sound, which means depending on theposition and direction of the glasses in an arena environment, the soundheard by the user through the glasses could be based on the direction,orientation and position in the arena of the user's glasses.

Accessories can be used in conjunction with glasses. Wirelessaccessories may include gun type devices to enable a user to engage inan action game by firing with a weapon in his hand, but utilizing thescreen, and speakers located on the glasses for feedback.

The discussion of high and low latency connection protocols andprocessing described elsewhere herein is also generally applicable todual-mode glasses. In addition, as dual-mode glasses may be used inconjunction with accessory controllers (e.g., a controller in the formfactor of a gun or paddle), the accessory controllers themselves mayhave dual-mode communication capability. In other embodiments, theaccessory controller may communicate only locally (e.g., using a lowlatency transmitter or transceiver) with the dual-mode eyeglasses, whichare then responsible for forwarding control commands and other datareceived from the accessory to the console or other game host.

FIG. 2F illustrates example dual-mode eyeglasses according to an exampleembodiment. The illustrated eyeglasses include a number of buttons,including a back button, a select button, volume up/down buttons, and apower on/off. The eyeglasses also include a trackball for mouse and/orscrolling control. The eyeglasses further include earbud speakers and adisplay in each of the eyepieces and processing module. The processingmodule includes a CPU and dual-mode RF transceivers. As noted, dual-modeeyeglasses are an example of a mobile device. Thus, the discussion withrespect to FIG. 4 applies equally to the illustrated dual-modeeyeglasses and its component parts.

Other embodiments may include additional input/output devices and/orsensors that are not illustrated in FIG. 2F. For example, the eyeglassesmay include position/orientation sensors, such as an accelerometer, GPSreceiver, gyroscope, or the like. Some eyeglasses may include one ormore cameras, such as a first camera facing outward and a second camerafacing inward and trained on one or both eyes of the user. In addition,other types of controls may be used, such as a touch pad (e.g., insteadof track ball), slider switch, or the like.

Example Processes

FIGS. 3.1-3.8 are example flow diagrams of processes performed byexample embodiments. The following flow diagrams illustrate dual-modecommunication processes performed by enhanced eyeglasses.

FIG. 3.1 is an example flow diagram of example logic for interactingwith an entertainment console. The illustrated logic in this and thefollowing flow diagrams may be performed by dual mode eyeglasses, suchas the mobile device 400 described with respect to FIG. 4, below. Moreparticularly, FIG. 3.1 illustrates a process 3100 that includesoperations performed by or at the following block(s).

At block 3101, the process performs facilitating interaction with theentertainment console via dual-mode eyeglasses that includes a display,a low latency transmitter communicatively coupled to the entertainmentconsole, and a high latency transceiver communicatively coupled to theentertainment console, by: performing operation(s) of block(s) 3102 and3103, described below.

At block 3102, the process performs transmitting control messages to theentertainment console via the low latency transmitter. Transmittingcontrol messages may include transmitting any message, signal, or datathat controls the operation of the entertainment console or somefunction or application being performed thereby. For example, a controlmessage may be a game control signal, such as up/down or left/right. Insome embodiments, if there is available bandwidth on the low latencyconnection, the process may elect to communicate other information ordata via this connection.

At block 3103, the process performs communicating data messages with theentertainment console via the high latency transceiver. Data messagesmay include game data including image or audio data. In otherembodiments, data messages may include video, audio, or metadata that isrelated to a video program being viewed via the entertainment console.

FIG. 3.2 is an example flow diagram of example logic illustrating anexample embodiment of process 3100 of FIG. 3.1. More particularly, FIG.3.2 illustrates a process 3200 that includes the process 3100, whereinthe facilitating interaction includes operations performed by or at oneor more of the following block(s).

At block 3201, the process performs sensing position and/or orientationof a head of a user who is wearing the dual-mode eyeglasses.

At block 3202, the process performs transmitting indications of theposition and/or orientation of the head of the user, the indiationstransmitted via the low latency transmitter to the entertainmentconsole. In some embodiments, the dual-mode eyeglasses include sensorsfor sensing positional information, such as an accelerometer, a GPSreceiver, a gyroscope, a magnetometer, a barometer, or the like. Thedual-mode eyeglasses may be configured to transmit information receivedfrom such sensors to the entertainment console or other system.

At block 3203, the process performs receiving feedback commands via thelow latency transmitter from the entertainment console, the feedbackcommands causing a feedback output device on the dual-mode eyeglasses toprovide feedback to the user, the feedback including at least one of asound, an image, or a vibration. In addition, the dual-mode eyeglassesmay include output devices, such as speakers, a display, a haptic outputdevice, or the like. The dual-mode eyeglasses may receive commands fromthe entertainment console that drive such output devices in order toprovide feedback to the user.

FIG. 3.3 is an example flow diagram of example logic illustrating anexample embodiment of process 3100 of FIG. 3.1. More particularly, FIG.3.3 illustrates a process 3300 that includes the process 3100, whereinthe facilitating interaction includes operations performed by or at oneor more of the following block(s).

At block 3301, the process performs receiving control messages from anaccessory device, the control messages received via a low latencyreceiver of the dual-mode eyeglasses. As noted, the dual-mode eyeglassesmay communicate with an accessory device (e.g., a paddle, a hand-heldcontroller, a gun) and/or act as an intermediary between the accessorydevice and the entertainment console.

At block 3302, the process performs forwarding the control messages fromthe accessory device to the entertainment console. In some embodiments,the dual-mode eyeglasses may translate the control messages from oneformat into another, such that they are recognizable to theentertainment console or other system

FIG. 3.4 is an example flow diagram of example logic illustrating anexample embodiment of process 3100 of FIG. 3.1. More particularly, FIG.3.4 illustrates a process 3400 that includes the process 3100, whereinthe facilitating interaction includes operations performed by or at oneor more of the following block(s).

At block 3401, the process performs causing the entertainment console tomodify a game model for an arena gaming environment based on controlmessages transmitted via the low latency transmitter, the arena gamingenvironment including multiple players each using dual-mode eyeglasses.As discussed above, enhanced mobile devices may be used in an arenagaming context. Here, the entertainment console (or other game system)may store or otherwise access a game model, such as a data structurethat includes a two or three-dimensional representation of the arenagame and the locations of the corresponding players/devices. This gamemodel may be updated based on location information and other sensor dataobtained from the player's dual-mode eyeglasses. The game model datastructure may include other fields or elements, such as for representinghealth or other status information about players, representing thelocation and other properties of non-living objects (e.g., chairs,walls, obstacles), and the like. The arena gaming environment may besituated in a room or other space (e.g., a hall or gymnasium). In theenvironment, the players may interact with one another in gameplay, suchas by engaging in a war game, athletic competition, capture the flag, orthe like. Managing the arena gaming environment may include facilitatinggameplay, receiving and storing information about players and objects inthe game/environment, transmitting game information to player devices,and the like.

At block 3402, the process performs receiving from the entertainmentconsole via the high latency transceiver game data based on the modifiedgame model. The dual-mode eyeglasses may receive from the entertainmentconsole game data, such as image data, audio data, user interfacecontrols, or the like. Typically, such data is transmitted via the highlatency (high bandwidth) connection, although some embodiments includeselection logic configured to use the low latency connection when it hasavailable bandwidth.

FIG. 3.5 is an example flow diagram of example logic illustrating anexample embodiment of process 3100 of FIG. 3.1. More particularly, FIG.3.5 illustrates a process 3500 that includes the process 3100, whereinthe facilitating interaction includes playing an interactive video gameexecuting on the entertainment console by operations performed by or atone or more of the following block(s).

At block 3501, the process performs transmitting gaming control messagesvia the low latency transmitter, wherein the low latency transmitter isa 2.4 GHz transmitter having a maximum range of 10 meters and a latencyof less than 10 milliseconds. In other embodiments, other ranges orlatencies are contemplated. In general, any latency that provides for aninteractive, real-time gaming experience may be acceptable, such as lessthan 5 ms, less than 10 ms, less than 15 ms, or the like. Because thedual-mode eyeglasses is typically in the same room as the entertainmentconsole, low range transmitters may be used, providing the additionalbenefit of low power consumption.

At block 3502, the process performs receiving gaming data via the highlatency transceiver, wherein the high latency transceiver communicatesusing IP packets via a Wi-Fi or cellular network. The IP communicationmay be via a private (e.g., a VPN or a restricted LAN) or open network(e.g., an open Wi-Fi network or the Internet).

FIG. 3.6 is an example flow diagram of example logic illustrating anexample embodiment of process 3100 of FIG. 3.1. More particularly, FIG.3.6 illustrates a process 3600 that includes the process 3100, whereinthe facilitating interaction includes viewing a video program receivedby the entertainment console, by operations performed by or at one ormore of the following block(s).

At block 3601, the process performs transmitting program selection andcontrol messages via the low latency transmitter, wherein the lowlatency transmitter is a 2.4 GHz transmitter having a maximum range of10 meters and a latency of less than 10 milliseconds. As noted above,various ranges and/or latencies are contemplated. In general, somewhathigher latencies may be acceptable for video program viewing. However,even in the video program context, relatively low latency (e.g., lessthan 30 ms, less than 40 ms) may be desirable to reduce the delayperceived by users who are channel surfing or interacting with anelectronic program guide or similar interface.

At block 3602, the process performs receiving program video data,program audio data, and/or program metadata via the high latencytransceiver, wherein the high latency transceiver communicates using IPpackets via a Wi-Fi or cellular network. As noted, IP communication maybe via a private (e.g., a VPN or a restricted LAN) or open network(e.g., an open Wi-Fi network or the Internet). Note that in someembodiments, program audio data may be transferred via the low latencytransmitter, in order to keep audio playback synchronized with videoimages. Furthermore, In some embodiments, video and/or audio data may betransmitted at differing resolutions or quality levels over the lowlatency transmitter and the high latency transceiver. For example, lowresolution video or lower quality audio may be transferred via the lowlatency transmitter, while a higher quality stream is concurrentlytransferred via the high latency transceiver. In this manner, thedescribed techniques can advantageously maintain interactivity byproviding the user with initial an video/audio signal with low latency,and then provide improved or supplemental video/audio over the highlatency channel.

At block 3603, the process performs presenting at least some of theprogram video data via the display of the dual-mode eyeglasses. Inaddition, or otherwise, the display of the dual-mode eyeglasses may beused to present metadata about the program, such as program information(e.g., actors, program summary, program schedule information),electronic program guides, or the like. For example, data forcommunication may be associated with a priority (e.g., 1, 2, 3) that maybe used by the process to determine which communication connection toutilize.

FIG. 3.7 is an example flow diagram of example logic illustrating anexample embodiment of process 3100 of FIG. 3.1. More particularly, FIG.3.7 illustrates a process 3700 that includes the process 3100, and whichfurther includes operations performed by or at the following block(s).

At block 3701, the process performs automatically selecting one of thelow latency transmitter or the high latency transceiver forcommunication with the entertainment console, based on the type oramount of data being communicated between the dual-mode eyeglasses andthe entertainment console. In some embodiments, the processautomatically decides whether to use the low latency transmitter or thehigh latency transceiver based on the type or amount of data to becommunicated (or already being communicated). For example, the processmay preferentially use the low latency transmitter until that connectionis saturated. In other embodiments, the data being communicated istagged or otherwise identified as being preferentially transmitted viaone or the other communication connection.

FIG. 3.8 is an example flow diagram of example logic illustrating anexample embodiment of process 3100 of FIG. 3.1. More particularly, FIG.3.8 illustrates a process 3800 that includes the process 3100, whereinthe low latency transmitter is a first low latency transmitter and thedual-mode eyeglasses includes a second low latency transmitter, andwhich further includes operations performed by or at the followingblock(s).

At block 3801, the process performs automatically determining tocommunicate via the second low latency transmitter instead of the firstlow latency transmitter, based in part on the amount of data beingcommunicated between the dual-mode eyeglasses and the entertainmentconsole via the first low latency transmitter. In some embodiments, thedual-mode eyeglasses includes multiple low latency transmitters that maytransmit using different protocols and/or frequencies. For example, thedual-mode eyeglasses may include a Bluetooth 2.4 GHz transceiver as wellas a 2.4 GHz game controller transmitter (or transceiver). As anotherexample, the dual-mode eyeglasses may include a Bluetooth transceiver aswell as an infrared transmitter. In some embodiments, the dual-modeeyeglasses may transmit control signals or other data using the firstlow latency transmitter until that transmitter has reached (or is near)capacity (e.g., bandwidth), at which point the dual-mode eyeglasses mayuse the second low latency transmitter for overflow. In otherembodiments, the dual-mode eyeglasses may attempt to substantiallybalance the amount of data being transmitted via the two transmitters.As another example, an application or other logic that is initiating thetransmission may explicitly request to use one or the other of the lowlatency transmitters.

Example Computing System Implementation

FIG. 4 is an example block diagram of an example computing system forimplementing example embodiments. In particular, FIG. 4 shows anenhanced mobile device 400 that may be utilized as the enhanced mobiledevice described with respect to FIG. 1. Note that a general purpose orspecial purpose computing system/processor, suitably instructed, may beused to implement (or as part of) mobile device 400.

In the embodiment shown, mobile device 400 comprises a computer memory(“memory”) 401, a display 402, one or more Central Processing Units(“CPU”) 403, Input/Output devices 404 (e.g., keyboard, touch screen, GPSreceiver, accelerometer, position sensor, and the like), a low latencytransceiver 405, and high latency transceiver 406. Dual-modecommunication logic 410 is shown residing in memory 401. In otherembodiments, some portion of the contents, some or all of the componentsof the logic 410 may be stored on and/or transmitted over the othercomputer-readable media 405. The logic 410 and any related componentspreferably execute on one or more CPUs 403 to perform the techniques orfunctions described herein. Other code or programs 430 (e.g., gamingapps, video streaming apps, and the like) and potentially other datarepositories, such as data repository 420, also reside in the memory401, and preferably execute on one or more CPUs 403. Note that one ormore of the components in FIG. 4 may not be present in any specificimplementation.

The mobile device 400 interacts with the consoles 455 (e.g., gamingconsoles, set-top boxes) via the low latency transceiver 405 and via thehigh latency transceiver 406. In the illustrated embodiment, thecommunication via the low latency transceiver 405 is direct. In otherwords, it does not pass through any intermediate systems or devices. Incontrast, the communication via the high latency transceiver 406 isindirect, in that it passes through the network 450. Note also that insome embodiments, the device 400 may not be capable of receiving datavia a low latency connection. In such cases, the device 400 would have atransmitter in place of the low latency the low latency transceiver 405.Furthermore, the device 400 may also or instead have one or more lowlatency connections to the consoles 455 via a wired connection, such asa USB cable. Note that while consoles are sometimes used as examples,the functions of a console may equivalently be incorporated orimplemented within a different form factor, such as within a smarttelevision, an all-home entertainment system, a distributed system, orthe like.

The network 450 may be or include the Internet and/or any networks usedto provide IP-based communication, such as a network based on Wi-Fi,Wi-Di, WiMAX, 3G, 4G, or the like. Wi-Fi may be based on standards suchas IEEE 802.11b, 802.11g, 802.11n over 2.4 GHz, 3.6 GHz, 5 GHz, or otherfrequencies. Wi-Di may be or include WirelessHD protocols. The network450 may be any combination of media (e.g., twisted pair, coaxial, fiberoptic, radio frequency), hardware (e.g., routers, switches, repeaters,transceivers), and protocols (e.g., TCP/IP, UDP, Ethernet, Wi-Fi, WiMAX)that facilitate communication between remotely situated humans and/ordevices.

The mobile device 400 may also interact with gaming servers 450 andthird-party systems/applications 460. The gaming servers 450 may includeservers or systems that are used to facilitate group or shared gaming.The third-party systems 460 may include servers or systems used forcontent distribution (e.g., media streaming servers, online musicstores, Web servers), application distribution (e.g., app stores), orthe like.

The logic 410 is shown executing in the memory 401 of the mobile device400. Also included in the memory are a user interface manager 415 and anapplication program interface (“API”) 416. The user interface manager415 and the API 416 are drawn in dashed lines to indicate that in otherembodiments, functions performed by one or more of these components maybe performed externally to the logic 410.

The logic 410 performs functions described herein. For example, thelogic 410 may format data for communication via the transceivers 405 and406. As another example, the logic 410 may automatically determine whichof the transceivers 405 and 406 to utilize, based on information aboutthe data being transmitted (e.g., type, amount, priority), informationabout the communication medium (e.g., whether the link is saturated ornot, link utilization level), information about the receiver (e.g.,requirements of the console 455), or the like.

The UI manager 415 provides a view and a controller that facilitate userinteraction with the logic 410 and its various components. For example,the UI manager 415 may provide interactive access to the logic 410, sothat uses can configure the operation of the dual-mode communicationcapability of the mobile device 400.

The API 416 provides programmatic access to one or more functions of thelogic 410. For example, the API 416 may provide a programmatic interfaceto one or more functions of the logic 410 that may be invoked by one ofthe other programs 430 or some other module. In this manner, the API 416facilitates the development of third-party software, such as userinterfaces, plug-ins, adapters (e.g., for integrating functions of thelogic 410 into mobile applications), and the like.

In addition, the API 416 may be in at least some embodiments invoked orotherwise accessed via remote entities, the gaming servers 450, theconsoles 455, and/or the third-party systems/applications 460, to accessvarious functions of the logic 410. For example, the console 455 mayselect a preferred communication frequency or channel (or set otherparameters) with respect to the low latency transceiver 405 via the API416.

In an example embodiment, components/modules of the logic 410 areimplemented using standard programming techniques. For example, thelogic 410 may be implemented as a “native” executable running on the CPU403, along with one or more static or dynamic libraries. In otherembodiments, the logic 410 may be implemented as instructions processedby a virtual machine that executes as one of the other programs 430. Ingeneral, a range of programming languages known in the art may beemployed for implementing such example embodiments, includingrepresentative implementations of various programming languageparadigms, including but not limited to, object-oriented (e.g., Java,C++, C#, Visual Basic.NET, Smalltalk, and the like), functional (e.g.,ML, Lisp, Scheme, and the like), procedural (e.g., C, Pascal, Ada,Modula, and the like), scripting (e.g., Perl, Ruby, Python, JavaScript,VBScript, and the like), and declarative (e.g., SQL, Prolog, and thelike).

The embodiments described above may also use either well-known orproprietary synchronous or asynchronous client-server computingtechniques. Also, the various components may be implemented using moremonolithic programming techniques, for example, as an executable runningon a single CPU computer system, or alternatively decomposed using avariety of structuring techniques known in the art, including but notlimited to, multiprogramming, multithreading, client-server, orpeer-to-peer, running on one or more computer systems each having one ormore CPUs. Some embodiments may execute concurrently and asynchronously,and communicate using message passing techniques. Equivalent synchronousembodiments are also supported. Also, other functions could beimplemented and/or performed by each component/module, and in differentorders, and by different components/modules, yet still achieve thedescribed functions.

In addition, programming interfaces to the data stored as part of thelogic 410, such as in the data store 420, can be available by standardmechanisms such as through C, C++, C#, and Java APIs; libraries foraccessing files, databases, or other data repositories; throughscripting languages such as XML; or through Web servers, FTP servers, orother types of servers providing access to stored data. The data store420 may be implemented as one or more database systems, file systems, orany other technique for storing such information, or any combination ofthe above, including implementations using distributed computingtechniques.

Different configurations and locations of programs and data arecontemplated for use with techniques of described herein. A variety ofdistributed computing techniques are appropriate for implementing thecomponents of the illustrated embodiments in a distributed mannerincluding but not limited to TCP/IP sockets, RPC, RMI, HTTP, WebServices (XML-RPC, JAX-RPC, SOAP, and the like). Other variations arepossible. Also, other functionality could be provided by eachcomponent/module, or existing functionality could be distributed amongstthe components/modules in different ways, yet still achieve thefunctions described herein.

Furthermore, in some embodiments, some or all of the components of thelogic 410 may be implemented or provided in other manners, such as atleast partially in firmware and/or hardware, including, but not limitedto one or more application-specific integrated circuits (“ASICs”),standard integrated circuits, controllers executing appropriateinstructions, and including microcontrollers and/or embeddedcontrollers, field-programmable gate arrays (“FPGAs”), complexprogrammable logic devices (“CPLDs”), and the like. Some or all of thesystem components and/or data structures may also be stored as contents(e.g., as executable or other machine-readable software instructions orstructured data) on a computer-readable medium (e.g., as a hard disk; amemory; a computer network or cellular wireless network or other datatransmission medium; or a portable media article to be read by anappropriate drive or via an appropriate connection, such as a DVD orflash memory device) so as to enable or configure the computer-readablemedium and/or one or more associated computing systems or devices toexecute or otherwise use or provide the contents to perform at leastsome of the described techniques. Some or all of the components and/ordata structures may be stored on tangible, non-transitory storagemediums. Some or all of the system components and data structures mayalso be stored as data signals (e.g., by being encoded as part of acarrier wave or included as part of an analog or digital propagatedsignal) on a variety of computer-readable transmission mediums, whichare then transmitted, including across wireless-based andwired/cable-based mediums, and may take a variety of forms (e.g., aspart of a single or multiplexed analog signal, or as multiple discretedigital packets or frames). Such computer program products may also takeother forms in other embodiments. Accordingly, embodiments of thisdisclosure may be practiced with other computer system configurations.

All of the above U.S. patents, U.S. patent application publications,U.S. patent applications, foreign patents, foreign patent applications,non-patent publications, and appendixes referred to in thisspecification and/or listed in the Application Data Sheet, including butnot limited to U.S. patent application Ser. No. 13/934,041, filed onJul. 2, 2013 and entitled “DUAL-MODE EYEGLASSES;” U.S. PatentApplication No. 61/667,261, filed on Jul. 2, 2012 and entitled“DUAL-MODE COMMUNICATION DEVICES AND METHODS FOR COMMUNICATING WITH ANENTERTAINMENT SYSTEM;” U.S. Patent Application No. 61/675,211, filed onJul. 24, 2012 and entitled “DUAL-MODE COMMUNICATION DEVICES AND METHODSFOR ARENA GAMING;” and U.S. Patent Application No. 61/682,668, filed onAug. 13, 2012 and entitled “DUAL-MODE EYEGLASSES;” are incorporatedherein by reference, in their entireties. This application is alsorelated by subject matter to U.S. patent application Ser. No. 13/934,035filed on the same day as this Application, and to U.S. patentapplication Ser. No. 13/934,039 also filed on the same day as thisApplication, both of which are incorporated herein by reference.

From the foregoing it will be appreciated that, although specificembodiments have been described herein for purposes of illustration,various modifications may be made without deviating from the spirit andscope of this disclosure. For example, the methods, techniques, andsystems for dual-mode communication are applicable to otherarchitectures or in other settings. For example, at least some of thetechniques may be employed in an educational setting, so that aclassroom of students may interact with a teacher and/or educationalsystem in a classroom or distributed setting. Also, the methods,techniques, and systems discussed herein are applicable to differingprotocols, communication media (optical, wireless, cable, etc.) anddevices (e.g., desktop computers, wireless handsets, electronicorganizers, personal digital assistants, tablet computers, portableemail machines, game machines, pagers, navigation devices, etc.).

The invention claimed is:
 1. An entertainment system comprising:wearable dual-mode eyeglasses including a translucent display, a lowlatency transmitter that is communicatively coupled to an entertainmentconsole, a high latency transceiver that is communicatively coupled tothe entertainment console, and a processing module, wherein the display,the low latency transmitter, the high latency transceiver, andprocessing module are integrated into the dual-mode eyeglasses, andwherein the entertainment console is a gaming console, personalcomputer, or a set top box communicatively coupled to a display andwherein the eyeglasses are configured to receive video data from theconsole via the high latency transceiver, the processing moduleconfigured to: transmit game control messages to the entertainmentconsole via the low latency transmitter; and communicate game data withthe entertainment console via the high latency transceiver; wherein atarget or menu item is selected by translating and transmitting headand/or eye movements via the low latency transmitter to theentertainment console and wherein game data transferred to theeyeglasses via the high latency transceiver is displayed on a secondarydisplay of the eyeglasses.
 2. The system of claim 1, wherein thedual-mode eyeglasses includes position sensors configured to sense theposition and/or orientation of a head of a user wearing the dual-modeeyeglasses, and wherein the module is further configured to transmitindications of the position and/or orientation of the head of the userto the entertainment console via the low latency transmitter.
 3. Thesystem of claim 1, wherein the dual-mode eyeglasses includes a lowlatency receiver configured to receive feedback commands from theentertainment console that cause a vibration signal to be presented. 4.A method for interacting with an entertainment console, the methodcomprising: facilitating interaction with the entertainment console viawearable dual-mode eyeglasses that includes a translucent display, a lowlatency transmitter communicatively coupled to the entertainmentconsole, a high latency transceiver communicatively coupled to theentertainment console, and a processing module, wherein theentertainment console is a gaming console, personal computer, or a settop box communicatively coupled to a display, wherein the eyeglasses areconfigured to receive video data from the console via the high latencytransceiver, and wherein the display, the low latency transmitter, thehigh latency transceiver, and processing module are integrated into thedual-mode eyeglasses, by: transmitting game control messages to theentertainment console via the low latency transmitter; and communicatinggame data messages with the entertainment console via the high latencytransceiver; wherein a target or menu item is selected by translatingand transmitting head and/or eye movements via the low latencytransmitter to the entertainment console and wherein game datatransferred to the eyeglasses via the high latency transceiver isdisplayed on a secondary display of the eyeglasses.
 5. The method ofclaim 4, wherein the facilitating interaction further comprises: sensingposition and/or orientation of a head of a user who is wearing thedual-mode eyeglasses; transmitting indications of the position and/ororientation of the head of the user, the indications transmitted via thelow latency transmitter to the entertainment console; and receivingfeedback commands via the low latency transmitter from the entertainmentconsole, the feedback commands causing a feedback output device on thedual-mode eyeglasses to provide feedback to the user, the feedbackincluding at least one of a sound, an image, or a vibration.
 6. Themethod of claim 4, wherein the facilitating interaction furthercomprises: receiving control messages from an accessory device, thecontrol messages received via a low latency receiver of the dual-modeeyeglasses; and forwarding the control messages from the accessorydevice to the entertainment console.
 7. The method of claim 4, whereinthe facilitating interaction further comprises: causing theentertainment console to modify a game model for an arena gamingenvironment based on control messages transmitted via the low latencytransmitter, the arena gaming environment including multiple playerseach using dual-mode eyeglasses; and receiving from the entertainmentconsole via the high latency transceiver game data based on the modifiedgame model.
 8. The method of claim 4, further comprising: automaticallyselecting one of the low latency transmitter or the high latencytransceiver for communication with the entertainment console, based onthe type or amount of data being communicated between the dual-modeeyeglasses and the entertainment console.
 9. The method of claim 4,wherein the transmitting control messages to the entertainment consolevia the low latency transmitter further comprises: transmitting controlmessages via an infrared communication module of the dual-modeeyeglasses.
 10. A non-transitory computer-readable medium includingcontents that are configured, when executed, to cause a computing systemto perform a method for interacting with an entertainment console, themethod comprising: facilitating interaction with the entertainmentconsole via wearable dual-mode eyeglasses that includes a translucentdisplay, a low latency transmitter communicatively coupled to theentertainment console, a high latency transceiver communicativelycoupled to the entertainment console, and a processing module, whereinthe entertainment console is a gaming console, personal computer, or aset top box communicatively coupled to a display, wherein the eyeglassesare configured to receive video data from the console via the highlatency transceiver, and wherein the display, the low latencytransmitter, the high latency transceiver, and processing module areintegrated into the dual-mode eyeglasses, by: transmitting game controlmessages to the entertainment console via the low latency transmitter;and communicating game data messages with the entertainment console viathe high latency transceiver, wherein a target or menu item is selectedby translating and transmitting head and/or eye movements via the lowlatency transmitter to the entertainment console and wherein game datatransferred to the eyeglasses via the high latency transceiver isdisplayed on a secondary display of the eyeglasses.
 11. The system ofclaim 1 wherein the target or menu item is selected responsive to a userlooking through the dual-mode eyeglasses.
 12. The system of claim 1further comprising: a soft pad or trackball attached to the dual-modeeyeglasses configured to indicate at least one of up, down, left, right,or tap buttons.
 13. The system of claim 1 further comprising: anaccessory device in communication with the dual-mode eyeglassesconfigured to generate the game control commands for transmitting ingame control messages to the entertainment console via the low latencytransmitter.
 14. The system of claim 13 wherein the accessory device isa handheld game controller or a shooting device.
 15. The system of claim13 wherein the game control commands include one or more of touch screencommands, keyboard commands, tilt, bank, or motion commands.
 16. Thesystem of claim 1 wherein game data communicated via the high latencytransceiver comprises at least one of metadata regarding a game beingplayed, a game map that updates as a player moves, sighting or aiming areal or virtual device that is part of the game, or 3D or motion sensingdata.
 17. The system of claim 1 wherein the game data communicated viathe high latency transceiver comprises metadata relevant to a moviebeing watched on a console connected display.
 18. The system of claim 1wherein the game data communicated via the high latency transceivercomprises camera data obtained from the eyeglasses used to map an arearound a user based on a visual setting of the user.
 19. The system ofclaim 1 wherein the dual-mode eyeglasses further comprise: a microphonecommunicatively coupled to the dual-mode eyeglasses wherein theeyeglasses are configured to communicate voice to the entertainmentconsole.
 20. The system of claim 1 wherein the dual-mode eyeglassesfurther comprise: speakers communicatively coupled to the dual-modeeyeglasses wherein the eyeglasses are configured to communicatestereophonic sounds.
 21. The system of claim 20 wherein the dual-modeeyeglasses are configured to provide position dependent surround soundbased upon the direction, orientation, and position of the eyeglasses.